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Abstract 

Background: Currently, 1 out of 88 children are diagnosed with an autism spectrum disorder (ASD), and the 
estimated cost for treatment services is $126 billion annually. Typically, ASD community providers (ASD-CPs) provide 
services to children with any severity of ASD symptoms using a combination of various treatment paradigms, some 
with an evidence-base and some without. When evidence-based practices (EBPs) are successfully implemented by 
ASD-CPs, they can result in positive outcomes. Despite this promise, EBPs are often implemented unsuccessfully 
and other treatments used by ASD-CPs lack supportive evidence, especially for school-age children with ASD. While 
it is not well understood why ASD-CPs are not implementing EBPs, organizational and individual characteristics 
likely play a role. As a response to this need and to improve the lives of children with ASD and their families, this 
study aims to develop and test the feasibility and acceptability of the Autism Model of Implementation (AMI) to 
support the implementation of EBPs by ASD-CPs. 

Methods/design: An academic-community collaboration developed to partner with ASD-CPs will facilitate the 
development of the AMI, a process specifically for use by ASD community-based agencies. Using a mixed methods 
approach, the project will assess agency and individual factors likely to facilitate or hinder implementing EBPs in 
this context; develop the AMI to address identified barriers and facilitators; and pilot test the AMI to examine its 
feasibility and acceptability using a specific EBP to treat anxiety disorders in school-age children with ASD. 

Discussion: The AMI will represent a data-informed approach to facilitate implementation of EBPs by ASD-CPs by 
providing an implementation model specifically developed for this context. This study is designed to address the 
real-world implications of EBP implementation in ASD community-based agencies. In doing so, the AMI will help to 
provide children with ASD the best and most effective services in their own community. Moreover, the proposed 
study will positively impact the field of implementation science by providing an empirically supported and tested 
model of implementation to facilitate the identification, adoption, and use of EBPs. 

Keywords: Autism spectrum disorder, Evidence-based practice, Implementation, Organization, Community provider, 
Model development 



Background 

Autism spectrum disorders (ASD) are a set of pervasive, 
clinically complex disorders that require multiple inter- 
vention types to alleviate common clinical targets, such 
as impaired social skills and communication, executive 
functioning, empathy and perspective taking, sensory 
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perception, and motor skills, along with restricted and 
circumscribed interests and co-occurring psychiatric dis- 
orders [1]. Currently 1 out of 88 children are diagnosed 
with an ASD [2], and treatments for ASD symptoms are 
estimated to cost $125 billion annually [3]. Typically, 
ASD community providers (ASD-CPs) provide services 
to children with any severity of ASD symptoms using a 
combination of various treatment paradigms, with some 
treatments considered efficacious or probably efficacious 
and some with no evidence-base [4-7]. When evidence- 
based practices (EBPs) are successfully implemented by 
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ASD-CPs, outcomes include increased IQ, communica- 
tion, social and daily living skills, and provide an 
improved developmental trajectory for children with 
ASD [8]. Despite this promise, EBPs are often imple- 
mented unsuccessfully, and other treatments used by 
ASD-CPs lack supportive evidence; both may be incon- 
sequential at best and harmful, even deadly, to children 
with ASD, at worst [9,10]. Given that 253% of California 
Department of Developmental Services' total budget 
pays for services for individuals with ASD [11], it is crit- 
ical that EBPs be successfully implemented by ASD-CPs 
to improve the lives of children with ASD and their fam- 
ilies, and as a measure of fiscal responsibility. 

Recently, research emphasis has been placed on devel- 
oping and testing EBPs for children with ASD and their 
families, resulting in an increase in the number of EBPs 
targeting many common clinical targets of the disorder 
[2,12]. However, ASD-CPs are not implementing these 
EBPs at the same rate that they are being developed and 
tested [5]. Instead, ASD-CPs are either delivering a 
single type of intervention for all clinical targets 
(e.g., Discrete Trial Teaching (DTT)), combining prac- 
tices in a non-systematic manner, or using practices that 
lack supportive evidence. For example, DTT is a highly 
structured EBP that targets ASD symptoms, such as 
communication, and has robust evidence supporting its 
use. However, it does not fit all children with ASD, in 
particular, higher functioning school-age children with 
ASD, nor treat many common clinical disorders experi- 
enced by children with ASD [6]. Thus, many children 
with ASD in the community are left with unmet clinical 
needs [7]. 

It is not well understood why ASD-CPs are not imple- 
menting EBPs for common clinical issues related to 
ASD. It is likely that both organizational and individual 
ASD-CP characteristics play a role in hindering imple- 
mentation. Further, preliminary individual interviews 
with a convenience sample of five ASD community 
agency leaders (herein referred to as ASD leaders') indi- 
cated a need for a contextually specific model designed 
for use by ASD-CPs to facilitate implementing EBPs for 
children with ASD. That is, while EBPs may be available, 
ASD-CPs do not have an efficient and effective process 
to facilitate EBP adoption, implementation, and sustain- 
ment. As a response to this need, and to improve the 
lives of children with ASD and their families, this study 
aims to develop the Autism Model of Implementation 
(AMI) that will facilitate implementation of EBPs within 
ASD community-based agencies. 

A contextually specific implementation model for 
ASD-CPs is necessary 

Current empirically derived implementation models 
[13-15] provide useful guidance for the development of 



the AMI given that they consider a comprehensive set 
of organizational and individual provider factors likely 
to contribute to the adoption and implementation of 
EBPs. However, it is not known whether these models can 
be generalized to ASD-CPs. Attempting to generalize em- 
pirically derived implementation models for other settings 
may fail to recognize the unique context and characteris- 
tics of ASD-CPs. For example, compared with other 
community-based contexts (e.g., community mental health 
providers), ASD-CPs differ in training, attitudes towards 
EBPs, and funding [5,16,17]. Preliminary data gathered for 
this study indicate that ASD Leaders are willing to adopt 
EBPs (after evaluating the EBP-agency fit) but identified 
implementation barriers ranging from the educational 
level of their staff (predominantly undergraduate college 
students), funding (e.g., need for ongoing intervention to 
treat pervasive symptoms, insurance limits treatment ses- 
sions and types of treatments, so many families must pay 
out-of-pocket for many services), societal constraints 
(e.g., parents may demand specific services that are not 
typically provided for specific symptoms of ASD), and 
difficulty identifying EBPs for use with children with 
ASD. ASD-CPs may be expected to adapt interventions 
to meet the needs of the heterogeneous set of symptoms 
and behaviors displayed by children with ASD without 
appropriate training or knowledge of the intervention. 
Further, parents and consumer advocacy groups have 
a strong influence on legislation and funding for ser- 
vices, thus affecting the organizational context of ASD 
agencies [18]. 

Collaboration with community partners 

Traditional efforts to implement EBPs in community set- 
tings have followed a research-to-practice approach to 
implementation, wherein researchers disseminate new 
EBPs to ASD-CPs in a unidirectional manner from the 
laboratory to community and expect practitioners to 
adopt, implement, and adhere to EBP protocols without 
it being tailored for the agency context or ASD-CPs 
characteristics, or inviting ASD-CPs to participate in the 
development of, and decision- making about the protocol 
[19]. Consequentially, efforts often result in failed imple- 
mentation, with providers reporting a lack of investment 
in the EBP and focusing on different needs than those 
being addressed by the EBP [20,21]. These challenges 
highlight the need for improved collaboration between 
academia and community stakeholders [22], Numerous 
collaborative partnership models exist, including 
community-based participatory research (CBPR) [21], 
community-participatory partnered research (CPPR) 
[23], participatory action research (PAR) [24], and 
academic-community collaboration (ACC) [19,25]. Be- 
cause this project is researcher-initiated, the ACC model 
will be used. The ACC model is characterized by: shared 
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vision and impact benchmarks, building interdepend- 
ence between collaborative partners, consensus and 
shared decision making, and formalized collaborative 
structure (e.g., roles, responsibilities of collaborators). 
ACC is an ideal approach to overcome previous imple- 
mentation challenges because it involves community sta- 
keholders as partners in most aspects of research 
endeavors [19,25] and will allow researchers to better 
address issues of external validity, feasibility, and accept- 
ability of the AMI. 

The autism model of implementation (AMI) 

While numerous empirically-derived models exist that 
are comprised of variables affecting adoption and imple- 
mentation of EBPs [13-15], no models focus on imple- 
mentation of EBPs by ASD-CPs, specifically. Further, 
few studies have conducted applied research to assess 
the outcomes of these models of implementation in 
practice. In the few studies investigating the application 
of implementation models [26,27], key elements leading 
to success have emerged, including: using distinct phases 
to guide implementation; involving direct service provi- 
ders and community members as collaborative partners 
providing input on the uptake of EBPs; and providing 
comprehensive training with ongoing support prior to 
and during early EBP uptake. 

The purpose of this study is to develop the AMI that 
will involve factors relevant to implementation on mul- 
tiple levels, including assessing both agency factors and 
provider characteristics related to facilitating or inhibit- 
ing implementation of EBPs (e.g., structured needs as- 
sessment; standardized process for evaluating the EBP, 
evaluating organizational adoption readiness factors). 
In collaboration with ACC partners, a review of the lit- 
erature and agency implementation assessment data 
will guide the development of the AMI. Preliminary 
discussions with ASD-CPs indicate that some factors 
included in general health service organizations' mod- 
els of implementation may be of particular importance 
to the implementation of EBPs by ASD-CPs (e.g., on- 
going support, flexibility of EBP) and require greater 
emphasis in the AMI. Furthermore, additional factors 
may need to be added to the AMI (e.g., assess need fac- 
tors). Although the exact factors included in the AMI 
will require completion of aim one research activities, 
Figure 1 presents a preliminary model that has been 
adapted in collaboration with ACC partners. In particu- 
lar, questions remain about a feasible model and 
process of implementation, such as: within an ASD 
community-based agency, who assesses client and 
agency need and how; who identifies the appropriate 
EBP and how; and what structures and processes can 
be embedded within the AMI regarding evaluating the 
validity, feasibility, or adaptability of the EBP? What 



agency-level training process will facilitate EBP implemen- 
tation? What structured evaluation of implementation is 
needed to sustain the use of the EBP by ASD-CPs over 
time? Results from aim one will be discussed with 
ACC partners in order to refine the preliminary AMI 
model. 

The work in the proposed project will proceed in three 
phases corresponding to three specific aims: 

• Aim one: Assess agency antecedents and readiness 
factors, individual ASD-CP characteristics, and 
methods of implementing EBPs by ASD Leaders to 
identify factors that facilitate or hinder adoption and 
use of EBPs by ASD-CPs. 

• Aim two: Develop the autism model of 
implementation (AMI) with the focus on 
identification, adoption, and use of EBPs within 
community-based agencies serving children 
with ASD. 

• Aim three: Examine the feasibility and acceptability 
of the AMI through a small-scale pilot study. It is 
hypothesized that ASD-CPs will successfully 
complete each activity within each phase of the 
AMI to implement new EBPs within their ASD 
community-based agencies. 

Methods/Design 

Aim one: Assess agency antecedents and readiness fac- 
tors, individual ASD-CP characteristics, and methods of 
implementing EBPs by ASD Leaders to identify factors 
that facilitate or hinder adoption and use of EBPs by 
ASD-CPs. 

Participants 

At least two focus groups involving approximately five 
to eight ASD-CPs will be conducted to assess perspec- 
tives related to EBP implementation factors. ASD-CPs 
will include ASD agency Leaders as well as direct service 
providers, and are expected to be mostly female and 
range in educational level from college students to Ph.D. 
level staff. Age is not known, but typically varies between 
24 and 69 years [5]. It is expected that ethnical/racial 
composition will reflect that of mental health service 
providers in San Diego, Imperial, and Orange Counties, 
which is diverse. 

Procedure 

Prior to the focus group, the principal investigator (PI) 
will conduct a comprehensive literature review and iden- 
tify theoretically derived implementation factors that are 
likely to be relevant for the AMI [13-15]. Once identi- 
fied, implementation factors will be further examined for 
applicability by using a mixed method research design 
involving quantitative measures (see below) and focus 
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Figure 1 Preliminary conceptual model 
et al., 2009 [14]; and Greenhalgh et al., 

(bolded and italicized). 



of the Autism Model of Implementation (Adapted from Aarons et al., 201 1 [13]; Damschroeder 
2004 [15]). Note. Discussions with ACC Partners yielded: factors to be emphasized (bolded) and added 



groups of ASD-CPs. Findings will be used to expand or 
narrow the focus of each theoretically derived imple- 
mentation factor. Specifically, focus group participants 
will be asked to complete and return the quantitative 
measures prior to the focus group in order to allow 
focus group discussions to elaborate on the responses. 
Focus groups will be convened at times convenient to a 
majority of participants to elicit perceptions of identified 
implementation factors as well as previously unidentified 
factors particular to ASD-CP services. Further, focus 
groups will evaluate what methods are being used, if 
any, to assess need, evaluate EBPs, make decisions 
regarding adoption, and facilitating implementation of 
EBPs. 

Measures 

Demographic survey 

The demographic survey will provide information 
about the ASD-CPs individual characteristics including 
gender, race/ethnicity, educational level, years as an 
ASD-CP at and outside of their current ASD agency, 
training received at the ASD agency, training received 
outside of the ASD agency, and professional or net- 
working organization membership. 

Evidence-based practice attitude scale (EBPAS) 

Focus group participants will complete the EBPAS 
[28,29], a structured questionnaire designed to assess 
empirically-derived constructs related to adoption and 
implementation of EBPs: appeal, requirements, open- 
ness, divergence, and a total EBPAS score representing 
the ASD-CPs global attitude toward EBPs [30,31]. 



Children's services survey (CSS) 

The CSS [32], adapted for use by ASD-CPs, will assess 
organizational culture and climate in order to provide 
information about factors related to implementing EBPs. 

ASD community provider strategies survey (PSS) 

The PSS, adapted for this study, will assess ASD-CPs' use 
of EBPs, knowledge of the evidence supporting the use of 
the practice, and whether they use the EBP or not, and 
whether the have received formal training in the EBP [33]. 

Focus group data 

Semi-structured focus group guides will be used with 
ASD-CPs to elicit data on their perspectives on the follow- 
ing topics: needs of both clients and ASD-CPs in regard 
to children with anxiety disorders; EBPs in general; agency 
and ASD-CP factors that might facilitate or inhibit EBP 
implementation; and their current process used to imple- 
ment EBPs (if any). Focus groups will be conducted until 
the data collected reaches saturation (Le., the same infor- 
mation is obtained from more than one group). The PI 
will introduce the initial issues to be addressed and mod- 
erate the dynamics of the group discussion to assure that 
all views are represented. This structure is dictated by the 
objective of collecting comparable data from both focus 
groups. Focus group discussions will follow a Tunnel inter- 
view structure' [34], starting with broader researcher- 
driven issues and narrowing to more participant-driven 
specific illustrations of these issues. 

Analysis plan: Integration of qualitative and quantitative 
analyses 

Integrating qualitative and quantitative methods involves 
both deductive and inductive approaches to the research 
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design and interpretation of data within the same study 
[35], and is most appropriate for addressing the com- 
plexities of modern social phenomena by utilizing mul- 
tiple approaches to understand a given phenomenon or 
process [36]. This study will employ a mixed method ap- 
proach in order to acquire understanding of the factors 
involved in implementation by ASD-CPs, and to evalu- 
ate the process of implementation that ASD-CPs are 
currently using in order to identify facilitating and inhi- 
biting factors related to implementing EBPs by ASD- 
CPs. Implementation factors are not easily quantified, 
requiring the use of qualitative methods and data to in- 
form the development of structured assessment and 
evaluation tools (quantitative measures) for future use. 
Three different strategies will be employed when inte- 
grating quantitative and qualitative data, as suggested by 
Creswell and Piano Clark [37]: triangulation, expansion, 
and complementarity. 

First, merging qualitative and quantitative data will 
occur through triangulation in which results of quantita- 
tive and qualitative analyses are placed side by side to de- 
termine whether each provides the same answer to the 
same question (convergence; e.g., do ASD-CP focus group 
data concur with quantitative data regarding attitudes 
regarding EBPs, organizational culture and climate, and 
EBP use and knowledge). Second, results from qualitative 
and quantitative analyses will be linked when the former 
is used to provide explanations for unanticipated findings 
produced by the latter (expansion; e.g., questionnaire data 
will be used to assess the prevalence of emergent barriers 
to implementing EBPs observed in qualitative interviews). 
Finally, results of qualitative analysis can be embedded 
within the analysis of quantitative data by helping to 
contextualize results obtained in statistical analyses fo- 
cused on the questionnaires (complementarity). 

Aim two: Develop the autism model of implementa- 
tion (AMI) with focus on identification, adoption, and 
use of EBPs within agencies serving children with ASD. 

Participants 

Development of the AMI will involve approximately 
seven ACC partners. The ACC will be comprised of three 
to five ASD-CPs who are responsible for directing, 
supervising, or training direct service providers within 
their agencies, one to two funding agency representa- 
tives (e.g., regional centers, insurance companies, school 
districts) who are knowledgeable about how funding deci- 
sions are made for children with ASD, and an implementa- 
tion researcher with expertise in studying implementation 
factors related to community agencies. A systematic re- 
cruitment strategy will be used in order to maximize the 
diversity of ACC partners, which is expected to increase 
the generalizability of the AMI to a variety of organiza- 
tions, geographic locations, and providers. 



Procedure 

In collaboration with ACC partners, the review of the 
literature and aim one data will guide the development 
of the AMI. Specifically, results will be presented to 
ACC partners to obtain feedback and additional inter- 
pretation. Results and ACC partner feedback will be 
used to guide the development of the AMI and AMI 
materials {e.g., needs assessment materials, et al.). Al- 
though the exact factors included in the AMI will re- 
quire completion of aim one research activities, the AMI 
will likely include four phases: assessing needs of children 
with ASD and the agency, examining EBP factors to ad- 
dress those needs, addressing organizational factors and 
ASD-CP characteristics related to adoption, and identify- 
ing implementation factors, such as training, EBP flexibil- 
ity, and ongoing support that will facilitate EBP use at the 
agency (Figure 1). For each phase, the AMI will have spe- 
cific products, developed by the PI in collaboration with 
ACC partners, to be used by ASD Leaders in order to ac- 
complish the goals of that phase. For example, ACC part- 
ners who collaborated in the development of this proposal 
reported that assessing need was an important step for an 
implementation model and that there was not a general- 
ized, systematic assessment tool available for use by ASD- 
CPs. Therefore, an organizational assessment tool to 
identify common clinical targets of school-age children 
with ASD will be developed to support the implementa- 
tion of the AMI. Table 1 outlines likely materials sup- 
porting AMI implementation. 

Process of developing the AMI 

The development of the AMI will involve an iterative 
approach (Figure 2) involving the ACC partners. Specif- 
ically, the PI will revise the preliminary conceptual 
model of the AMI based on the literature review and 
aim one results. The ACC partners will provide feedback 
to enhance the interpretation of aim one results and for 
revisions to the AMI and AMI process materials. It is 
expected that this iterative approach will increase the ex- 
ternal validity, feasibility, and acceptability of the AMI. 

Aim three: Examine the feasibility and acceptability of 
the AMI through a small-scale pilot study. 

The AMI model will be designed for use by ASD 
community-based agencies to implement any EBP. How- 
ever, for this pilot study, an EBP has been chosen a 
priori. This will allow for examination of each phase of 
the AMI and preliminary comparison between use of the 
AMI among agencies who participate in the pilot study. 
Specifically, an EBP to treat co-occurring anxiety disor- 
ders in children with ASD has been targeted due to the 
prevalence of anxiety disorders among school-age chil- 
dren with ASD and interest expressed by ACC partners. 

Clinically significant anxiety disorders affect up to 80% 
of school-age children with ASD, causing heightened 
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Table 1 AMI Process Materials for the AMI preliminary conceptual model 



AMI Phase 



AMI Process Materials 



Phase 1: Assessing Need 



Phase 2: EBP Factors 



Phase 3: Adoption Factors 
(including preparing for uptake) 



Phase 4: Implementation Factors 



• Assessment to identify common clinical targets of school-age children with ASD. 

• Assessment to identify agency need. 

• Guidance for locating EBPs to meet common clinical targets of school-aged children with ASD through 
targeted search engines cataloguing EBPs, such as National Standards Project, NREPP, Promising Practices 
Network, and PracticeWise. 

• Structured evaluation process to identify the goals, content, and structure of the EBP, rate the 
adaptability of the EBP, and training requirements. 

• Structured guide for evaluating the validity/evidence supporting the use of the EBP. 

• Recommendations for enhancing agency antecedents and readiness for adoption of EBPs. 

• Assessing fit between goals of the EBP and agency values/mission. 

• Structured tool for assessing the feasibility of the EBP for use by ASD-CPs. 

• Process for adapting the EBP for use by ASD-CPs without reducing effectiveness, as needed. 

• Decision tree involving evaluation of the implications of adoption including personnel, 

dedicated time/resources, initial training, ongoing support, materials cost, etc. (e.g., cost-benefit calculator). 

• Structured staff activity related to assessing ASD-CP motivation, existing skills and capacity, 
attitude toward EBP, and fit. 

• Planning tools for update of EBPs including training, ongoing support, adapting EBP, and fidelity 
monitoring of EBP use. 

• Re-evaluation of needs assessment. 



impairment above and beyond that caused by the ASD, 
and is the second most highly prevalent problem 
reported by parents [38-43]. Cognitive behavioral therapy 
(CBT) to treat anxiety disorders in children with 
ASD is considered evidence-based because multiple 
laboratories using rigorous research designs and 
methods (e.g., randomized controlled trials) have found 
evidence supporting its use to alleviate or reduce anxiety 
symptoms or interference with large treatment effect sizes 
at post- treatment compared to control conditions [44-46] . 



Moreover, evidence suggests that intervening on chil- 
dren's anxiety disorders has distal outcomes such as in- 
creasing social and daily living skills, and family 
functioning [44,47,48]. Preliminary data gathered from 
ASD Leaders indicate that anxiety is a significant prob- 
lem for some of the school-age children being provided 
services at ASD community-based agencies. Interviews 
with ASD Leaders regarding current use of CBT by 
ASD-CPs found only two of the five agencies using CBT 
strategies to treat anxiety among school-age children 



Investigator & 
Collaborators 



Intervention 
Development 



Intervention 
Implementation 



Outcomes 



PI Activities: 

• Synthesis of literature 

• Collect data 

• Data analyses and data 
interpretation 

• Develop AMI 

• Co-author manuscripts 



ACC Partners: 

• Feedback/guidance on literature 
synthesis 

• Interpret data analyses 

• Feedback on AMI 

• Co-author manuscripts 



Autism Model of Implementation 
and AMI Process Materials 




Implementation of the AMI using ^ 
a specific EBP ("Building 
Confidence") to treat anxiety 
disorders in school age children 
with ASD. 

Participants include ASD Leaders 
(n«10) j 



• Change in EBPAS, CSS, PSS 
from pre-to post- 
implementation (Quant) 

• Use and fidelity measures 
(Quant) 

• SIC (Quant) 

' Perceived acceptability of the 
AMI (Qual) 



Figure 2 Process model of AMI development and pilot test. 
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with ASD. Of these, one agency reported using a man- 
ualized anxiety CBT developed for typically developing 
children with anxiety disorders rather than developed 
for children with ASD, and the other used exposure 
therapy (a component of CBT) but indicated that ASD- 
CPs were not certain about how to effectively use this 
practice with children with ASD. Using the AMI to 
implement a specific EBP to treat anxiety disorders 
in school-age children with ASD, aim three will 
evaluate the: feasibility of using the AMI within ASD 
community-based agencies; perceived fit, relevance, utility 
and compatibility of the AMI within ASD community- 
based agencies; training and support in the use of the AMI 
and AMI process materials; and observed fidelity of using 
the AMI to implement at EBP within ASD community- 
based agencies. 

Participants 

Approximately 10 ASD Leaders (e.g., clinical directors, 
executive directors), not participating in the ACC, will be 
recruited to participate in the AMI pilot study to test the 
feasibility and acceptability of the AMI. An agency will 
be eligible if it provides behavioral interventions, speech 
and language therapy, or social skills training to children 
with ASD and co-occurring anxiety disorders between the 
ages of 7-11 years. Agencies will be excluded if they do 
not provide services to children aged 7-11 years or do not 
serve any children with co-occurring anxiety disorders. 

Procedure 

The AMI will be used to implement 'Building Confi- 
dence,' a manualized CBT for children with ASD and co- 
occurring anxiety disorders [49]. The PI is a co-author 
of the manualized intervention and will consult with the 
primary author throughout the pilot study to address 
EBP factors (e.g., time available for training, billing pro- 
cedures, et ai), organizational adoption-related factors, 
and factors facilitating implementation. The pilot test 
will involve a pre-post design. ASD Leaders will be asked 
to complete questionnaires and participate in interviews 
prior to and after implementation of the AMI. Measures 
of AMI implementation will involve process and fidelity 
measures within each agency. 

Data/measures 

Data from the demographic survey, EBPAS, CSS, and 
PSS used during aim one will be administered at pre- 
and post-implementation of the AMI (see aim one for 
more information about these measures). 

Stages of implementation completion (SIC) 

The SIC [50] is a time-to-event based observational 
measure of implementation progress that consists of eight 
stages relating to specific implementation milestones. 



These milestones include: engagement, consideration of 
feasibility, readiness planning, staff hired/trained, adher- 
ence monitoring processes in place, services and consult- 
ation begin, ongoing services, consultation fidelity 
monitoring and feedback, and competency. The mile- 
stones span the timeframe from the engagement stage 
where initial contact between interested parties occurs 
through the attainment of program competency stage. 
Stages extend from ASD Leaders through ASD-CPs in 
order to capture the multiple levels involved in imple- 
mentation. The SIC is intended to be tailored to fit spe- 
cific EBPs and implementation models, and can be 
modified to assess quality and fidelity of the stages of 
implementing the AMI. The measure authors have 
agreed to work with the PI to adapt the SIC to best re- 
flect the phases and steps within the AMI. Analysis of 
the SIC involves a time-to-event modeling method. 

Individual interviews 

Leaders will be interviewed prior to the AMI implemen- 
tation to assess their perceptions of the AMI, including 
perceived fit, relevance, utility, and compatibility with 
their ASD agency. At post-implementation, interviews 
will assess perspectives about the AMI, including feasibil- 
ity and acceptability, changes in agency process for 
implementing EBPs, and changes in ASD-CP practices. 
Individual interviews will take approximately 30 minutes. 

Analysis plan: Integration of qualitative and quantitative 
analyses 

Repeated measures ANOVA for pre- and post- 
implementation questionnaires will be used to examine the 
changes that occurred during AMI. Due to the lack of a 
control group and the small sample sizes, interpretation of 
effect sizes will be examined. 

The SIC will be analyzed with guidance from the 
measure authors and statistical consultant using the Cox 
proportional hazards time-to-event modeling method 
[51] within multiple stages. Analyses will replicate those 
used during the measure development. In particular, an 
outside observer (e.g., research assistant) will track the 
dates of completion of each activity within each phase of 
the AMI in a stage-tracking database. From these data, 
three scores are derived: the number of AMI phases 
completed, the time spent in each AMI phase, and the 
proportion of AMI Process Materials completed in each 
phase. For sites that choose to discontinue implementa- 
tion, the discontinuation date will be logged accordingly 
in the furthest phase that the site entered. If this occurs, 
the time spent in the final phase of the AMI is calculated 
between the date of the earliest activity within the phase 
and the date of discontinuance. If the observation period 
ends before the phase is completed but a site has not 
discontinued implementation, the observation will be 
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treated as being censored (e.g., standard time-to-event 
analysis). 

Pre- and post-implementation interviews will be ana- 
lyzed using a coding, consensus, and comparison meth- 
odology [52], which follow an iterative approach rooted 
in grounded theory [53]. Transcriptions will be inde- 
pendently coded by the PI and a trained research assist- 
ant at a general level in order to condense data into 
analyzable units. The coding will be reviewed by grant 
mentors and consultants. Transcriptions will be assigned 
codes based on a priori or emergent themes. Transcrip- 
tions may be assigned more than one code. Disagree- 
ments will be resolved through subsequent discussion 
between research team members. The final list of codes 
will be developed through consensus with ACC partners. 
Based on the codes, the process of axial coding will be 
used to generate a series of categories arranged in a tree- 
like structure connecting segments into separate cat- 
egories or nodes. Nodes and trees will be used to create 
a taxonomy of themes, including a priori and emergent 
categories and new, unrecognized categories. Results will 
be presented to ACC partners for interpretation. 

Finally, methods for integrating qualitative and quanti- 
tative data for analyses will be the same as those used in 
aim one (see aim one for details). 

Discussion 

The AMI will represent a data-informed approach to fa- 
cilitate implementation of EBPs by ASD-CPs by providing 
an implementation model specifically developed for this 
context. This study is designed to address the implications 
of EBP implementation in ASD community-based agen- 
cies. In doing so, the exploratory and developmental work 
proposed in this study aims have a positive public health 
impact on the lives of children with ASD and their fam- 
ilies by increasing the implementation of EBPs by ASD- 
CPs, especially for children with ASD with co-occurring 
anxiety disorders. Moreover, the proposed study will posi- 
tively impact the field of implementation science by pro- 
viding an empirically-supported and tested model of 
implementation to facilitate the identification, adoption, 
and use of EBPs by ASD-CPs, specifically. 

In this project, the AMI will be developed and pilot 
tested. The AIM is designed to assist ASD community- 
based agencies assess and find EBPs to their meet agency 
needs, systematically evaluate the EBP for acceptability 
and feasibility within the agency, consider salient adop- 
tion factors, and guide the effective implementation of 
the EBP. Through the use of the AMI phases and AMI 
process materials (e.g., structured materials guiding the 
phases of implementation), it is hypothesized that imple- 
menting EBPs will be more likely for ASD community 
agencies. 



Once developed and pilot tested, further refinement of 
the AMI, including specific aspects of the AMI process 
materials, will be done to allow for larger-scale imple- 
mentation studies focusing on testing the implementa- 
tion effectiveness, generalizability, and sustainability of 
the AMI. By continuing to involve ACC partners, the ex- 
ternal validity of refinements made to the AMI will in- 
crease. Additionally, larger-scale studies will allow for 
involving more varied organizations or systems providing 
services to children with ASD (e.g., school districts) as 
well as diverse providers, and multilevel data (e.g., ASD- 
CPs nested within managers nested within agencies 
nested within service systems). Future tests of the ef- 
fectiveness of the AMI will not have an EBP selected a 
priori. 
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